Within the area of image processing thermal images, also called IR images, these images suffer from multiple deficiencies and are usually noisy, have a low resolution, have low contrast, and have low brightness, which makes it difficult to identify objects in the image, thus requiring preprocessing to improve their quality. A thermal image depicting a scene can be contrast-enhanced by combining the thermal image with a High Spatial Frequency Content (HSFC) image obtained from image information from a visual (or visible) light (VL) image, said combination being known as contrast-enhanced fusion or blending. A number of technical problems may arise when attempting to accomplish such combination and contrast enhancement.
A thermal imaging device, such as an IR camera, may be provided together with a VL camera to capture a visual light image and a thermal image and to process these images so that they can be displayed together as a contrast-enhanced image. The combination is advantageous in identifying variations in temperature in an object using IR data or thermal data from the thermal image while at the same time displaying a sufficient amount of data from the visual light image to simplify orientation and recognition of objects in the resulting image to a user using the camera. This may be achieved, for example, by enhancing the luminance component of IR image data values/pixel values with luminance components representing edges in the visual light image.
A High Spatial Frequency Content (HSFC) image typically comprises edges and corners derived from visual light image information, e.g. by high pass filtering the visual light image. The HSFC image and IR image are then combined by combining luminance information or luminance components of each pixel of the aligned HSFC image and each pixel of the aligned IR image into a contrast-enhanced combined image, thereby improving the low contrast in the thermal image. A method and apparatus for combining a thermal image and a HSFC image is disclosed in U.S. Pat. No. 8,520,970 B2 issued Aug. 27, 2013 and assigned to FLIR Systems AB, which is hereby incorporated by reference in its entirety.
A possible problem when applying such a technique is a parallax error between the thermal and visual light imaging systems. The capturing of the thermal image and the visual light image is performed by separate thermal and visual light imaging systems of the camera. The optical axes of the respective imaging systems may for example be configured with an offset from each other and configured with a relative angle between the optical axes. Thus, an optical phenomenon known as parallax error will arise that is more predominant when capturing images of a scene at a close range, e.g. less than 10 m between the depicted object and the thermal imaging device. To eliminate or reduce the parallax error, the images must be aligned. A large parallax error may result in ghosting or a ghost contrast-enhanced combined image, where the High Spatial Frequency Content (HSFC) image is offset in relation to the thermal image and is combined, e.g. super-imposed on top of the thermal image.
In some conventional image alignment techniques, the images are aligned based on a focusing distance derived from a position of a focusing arrangement, such as a focus motor, configured for determining a distance to an object being captured by the imaging parts. However, if such an alignment technology is not used and/or under certain circumstances there may be a remaining parallax error that causes the mentioned ghosting phenomena.
Yet another problem may occur when the imaged scene does not comprise any edges or corners, and the contrast-enhanced combined image therefore is not improved. That is, in this case the HSFC image then primarily comprises noise and thus the contrast of the combined image is not improved. On the contrary, image quality and interpretability of a combined image being processed in this manner for enhancing contrast may even be reduced.
Yet another problem is that some parts of the scene comprise edges or corners while other parts of the scene lack edges or corners, thereby causing reduced image quality and interpretability of the contrast-enhanced combined image in a part or a subset of the image.
Yet another problem may occur when an imaging system, e.g. the thermal imaging system, is out-of focus. This may be the case when a focusing distance range, i.e. a minimum acceptable focusing distance to the maximum acceptable focusing distance, of the imaging systems does not overlap with the distance to the scene. This will result in an increased parallax error or offset in the image alignment processing, resulting in a deviation between pixels representing the same object in the HSFC image and the thermal image. This may in its turn cause ghosting or a ghost image, where the HSFC image is offset in relation to the thermal image and is combined, for example super-imposed on top of the thermal image.
Yet another problem may occur in cases when some parts of the scene are within the imaging system focusing range while other parts of the scene are not, thereby causing ghosting in a part or subset of the image.
Yet another problem may occur when there is too little or too much light present for obtaining a good quality visual light image. When it is too dark, the visual light image may contain more noise than information. When it is too bright, the visual light image may contain too little contour/edge information.
Thus, there exists a need for an improved way of providing a contrast-enhanced combined image comprising data from a thermal image and data from a High Spatial Frequency Content (HSFC) image.